Bias Reduction in Instrumental Variable Estimation through First-Stage Shrinkage

The two-stage least-squares (2SLS) estimator is known to be biased when its first-stage fit is poor. I show that better first-stage prediction can alleviate this bias. In a two-stage linear regression model with Normal noise, I consider shrinkage in the estimation of the first-stage instrumental variable coefficients. For at least four instrumental variables and a single endogenous regressor, I establish that the standard 2SLS estimator is dominated with respect to bias. The dominating IV estimator applies James–Stein type shrinkage in a first-stage highdimensional Normal-means problem followed by a control-function approach in the second stage. It preserves invariances of the structural instrumental variable equations. Introduction The standard two-stage least-squares (2SLS) estimator is known to be biased towards the OLS estimator when instruments are many or weak. In a linear instrumental variables model with one endogenous regressor, at least four instruments, and Normal noise, I propose an estimator that combines James– Stein shrinkage in a first stage with a second-stage control-function approach. Jann Spiess, Department of Economics, Harvard University, [email protected]. I thank Gary Chamberlain, Maximilian Kasy, and Jim Stock for helpful comments. 1 Unlike other IV estimators based on James–Stein shrinkage, my estimator reduces bias uniformly relative to 2SLS. Unlike LIML, it is invariant with respect to the structural form and translation of the target parameter. I consider the first stage of a two-stage least-squares estimator as a highdimensional prediction problem, to which I apply rotation-invariant shrinkage akin to James and Stein (1961). Regressing the outcome on the resulting predicted values of the endogenous regressor directly would shrink the 2SLS estimator towards zero, which could increase or decrease bias depending on the true value of the target parameter. Conversely, shrinking the 2SLS estimator towards the OLS estimator can reduce risk (Hansen, 2017), but increases bias towards OLS. Instead, my proposed estimator uses the first-stage residuals as controls in the second-stage regression of the outcome on the endogenous regressor. If no shrinkage is applied, the 2SLS estimator is obtained as a special case, while a variant of James and Stein (1961) shrinkage that never fully shrinks to zero uniformly reduces bias. The proposed estimator is invariant to a group of transformations that include translation in the target parameter. While the limited-information maximum likelihood estimator (LIML) can be motivated rigorously as an invariant Bayes solution to a decision problem (Chamberlain, 2007), these transformations rotate the (appropriately re-parametrized) target parameter and invariance applies to a loss function that has a non-standard form in the original parametrization. In particular, unlike LIML, the invariance of my estimator applies to squared-error loss. The two-stage linear model is set up in Section 1. Section 2 proposes the estimator and establishes bias improvement relative to 2SLS. Section 3 develops invariance properties of the proposed estimator.